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FOREWORD 


The  U.S.  Army  Research  Institute  (ARI)  Armored  Forces  Research  Unit  at  Fort  Knox  is 
responsible  for  research  and  development  on  armor  training  and  simulation.  In  conjunction  with 
the  Simulation  Training  and  Instrumentation  Command  Project  Manager  (PM)  for  the  Combined 
Arms  Tactical  Trainer  (CATT)  and  the  Training  and  Doctrine  Command  Systems  Manager 
(TSM)  for  CATT,  ARI  contracted  to  develop  structured  exercises  and  training  support  packages 
for  use  in  the  Initial  Operational  Test  and  Evaluation  (lOTE)  of  the  Close  Combat  Tactical 
Trainer  (CCTT).  The  CCTT  is  the  first  system  developed  as  part  of  the  CATT  family  of  unit 
trainers.  The  CCTT  system  provides  a  simulated  environment  for  training  armor,  armored 
cavalry,  mechanized  and  infantry  units.  During  a  Limited  User  Test  (LUT)  of  CCTT  at  Fort 
Hood,  TX  in  preparation  for  the  lOTE,  ARI  and  the  contractor  monitored  the  initial  trial 
implementation  to  formatively  evaluate  the  developed  exercises  and  training  support  packages. 

ARI  conducted  this  research  pursuant  to  a  Memorandum  of  Agreement  with  the  U.S. 
Army  Armor  Center  and  Fort  Knox:  Manpower,  Personnel  and  Training  Research,  Development, 
Test,  and  Evaluation  for  the  Mounted  Forces,  18  October  1995.  The  ARI  Armored  Forces 
Research  Unit  performed  this  research  as  part  of  Research  Task  2124:  Strategies  for  Training  and 
Assessing  Armor  Commander’s  Performance  with  Devices  and  Simulations  (STRONGARM). 

The  present  research  report  identifies  management  procedures  and  tools  needed  to 
maintain  and  increase  the  effectiveness  and  efficiency  of  structured  training  with  the  CCTT.  The 
report  based  its  findings  on  observations  during  the  LUT  and  previous  experience  with  the 
Virtual  Training  Program  using  the  Simulation  Networking  (SIMNET)  system  at  Fort  Knox  with 
similar  structured  training  exercises  and  support  packages.  The  authors  provided  the  information 
in  this  report  to  the  PM-CATT  and  TSM-CATT,  and  presented  its  findings  to  the  1997  Fall 
Simulation  Interoperability  Workshop  in  Orlando,  FL.  This  workshop  series  sponsored  by  the 
Simulation  Interoperability  Standards  Organization  develops  standards  and  practices  supporting 
the  interoperability  and  reuse  of  models  and  simulations.  The  report  will  be  useful  to  simulation 
and  training  system  developers,  training  simulation  managers,  and  others  implementing 
structured  training  programs. 


ZITA  M.  SIMUTIS 
Technical  Director 
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SUSTAINING  AND  IMPROVING  STRUCTURED  SIMULATION-BASED  TRAINING 
EXECUTIVE  SUMMARY  _ _ _ _ 


Research  Requirement: 

The  U  S.  Army  Research  Institute  (ARI)  Armored  Forces  Research  Unit  at  Fort  Knox  has 
developed  a  structured  training  methodology  for  exploiting  the  use  of  virtual  (e.g.,  Simulation 
Networking-SIMNET)  and  constructive  (e.g.,  Janus)  simulations.  Structured  exercises  and 
training  support  package  (TSP)  materials  developed  for  the  Virtual  Training  Program  (VTP)  at 
Fort  Knox  have  proved  to  be  effective,  enabling  efficient  use  of  these  simulations.  This 
methodology  was  then  applied  to  develop  a  limited  set  of  similar  structured  exercises  and  TSP 
materials  for  the  Close  Combat  Tactical  Trainer  (CCTT).  The  Combined  Arms  Tactical  Trainer 
(CATT)  program,  directed  by  the  U.  S.  Army  Simulation,  Training,  and  Instrumentation 
Command  Project  Manager  (PM-CATT),  is  developing  CCTT  as  the  first  system  in  a  family  of 
simulations  designed  to  meet  established  unit  collective  training  requirements.  The  CCTT  and 
CATT  are  central  elements  of  the  Combined  Arms  Training  Strategy  that  emphasizes  simulation 
training  as  a  key  to  maximizing  the  benefits  of  limited  unit  field  training  resources.  Simulation 
training  prepares  units  for  field  training  as  well  as  helping  to  sustain  unit  proficiency  after  field 
training.  The  Initial  Operational  Test  &  Evaluation  of  the  CCTT  will  examine  system  training 
effectiveness  using  the  structured  training  exercises  and  TSP  materials  developed  by  ARI,  and 
reviewed  and  approved  by  the  Training  and  Doctrine  Command  System  Manager  (TSM-CATT). 
Research  on  management  and  delivery  of  structured  training  contributes  to  the  long-term 
effectiveness  of  the  CCTT  training  program  as  implemented  by  Army  units. 

Procedure: 


ARI  and  contractor  personnel  monitored  the  initial  trial  implementation  of  the  developed 
exercises  and  TSP  materials  during  the  CCTT  Limited  User  Test  (LUT).  The  Test  and 
Experimentation  Command  (TEXCOM)  conducted  this  test  at  Fort  Hood,  TX  to  plan  and 
prepare  for  the  lOT&E.  Observed  problems  were  noted  and  discussed  with  site  Contractor 
Logistical  Support  (CLS)  personnel,  unit  trainers  and  leaders,  TEXCOM  observers,  and 
representatives  of  the  PM-CATT  and  TSM-CATT.  The  contractor  obtained  comments  that 
raised  additional  issues  from  interviews  and  surveys  conducted  as  part  of  the  formative 
evaluation  of  the  TSPs  conducted  in  conjunction  with  the  LUT .  ARI  selectively  reviewed 
published  literature  on  quality  management  methods,  and  interviewed  points  of  contact  provided 
by  the  Fort  Knox  Office  of  Total  Army  Quality. 

Findings: 

Management  and  execution  of  structured  training  with  the  CCTT  presents  a  number  of 
significant  challenges.  In  part,  the  system  design  engenders  these  challenges  by  controlling  an 
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exercise  through  operation  of  a  number  of  interacting  workstations.  Furthermore,  unlike  the 
VTP  with  SIMNET  and  Janus,  there  is  no  established  group  of  observer/controllers  dedicated  to 
operator  duties  and  supervising  training.  Manning  the  workstations  takes  a  combination  of  CLS 
personnel  provided  by  the  CCTT  site,  and  Army  trainers  and  operators  provided  by  the  training 
unit’s  superior  headquarters  or  sister  units.  The  Army  personnel  require  considerable  preparatory 
training  and  rehearsal  to  perform  their  roles  properly  in  structured  training  exercises. 

Several  other  threats  to  the  effectiveness  of  CCTT  include:  (a)  divided  administrative 
responsibilities  requiring  coordination  among  the  installation,  CCTT  site,  and  training  unit,  (b) 
competing  demands  on  Army  personnel  making  it  difficult  to  provide  qualified  trainers  and 
operators  with  sufficient  time  for  preparation,  (c)  complex  procedures  for  preparation  and 
training  leading  to  incomplete  or  imperfect  execution,  (d)  long  lead  times  required  for  any 
substantial  exercise  modifications  desired  by  units,  (e)  systems  limitations  and  effort  required  to 
modify  exercises  required  by  CCTT  changes  in  hardware  and  software,  and  (f)  unmet  needs  for 
new  exercise  development  associated  with  long-term  changes  in  Army  equipment  and  doctrine 
(e.  g.,  digital  operations). 

Conceived  in  a  Total  Quality  Management  (TQM)  framework,  this  report  proposes 
processes  designed  to  sustain  and  improve  the  training  effectiveness  of  the  CCTT  throughout 
system  life.  The  emphasis  is  on  continuous  monitoring  of  training  process  and  product  indicators 
to  provide  management  feedback,  and  establishing  process  action  teams  to  identify  and  solve 
system  problems.  The  Army  has  adopted  TQM  as  part  of  its  management  philosophy,  and  has 
established  a  Total  Army  Quality  (TAQ)  program  with  offices  at  most  installations.  However, 
the  TAQ  program  has  addressed  few  training  issues.  Installation  commanders  could  employ 
some  TAQ  assets  to  foster  continuous  improvement  in  CCTT  training.  The  report  identifies 
some  training  quality  indicators,  and  management  support  tools  that  should  network  and  operate 
with  both  the  CCTT  and  existing  Army  training  management  software  and  databases. 

Utilization  of  Findings: 

Installation  commanders  and  site  managers  can  use  this  report  to  improve  management 
and  implementation  of  CCTT  structured  training.  The  findings  should  prove  to  be  generally 
useful  to  developers  of  simulator  systems,  and  developers  of  structured  exercises  and  TSPs  for 
simulator  training  programs.  The  information  in  this  report  should  contribute  to  interoperability 
standards  for  the  simulation  Federation  Development  Process.  The  report  also  suggests 
standards  for  training  management  tools  required  in  future  simulations  that  conform  to  the  High 
Level  Architecture  mandated  by  the  Defense  Modeling  and  Simulation  Office. 
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SUSTAINING  AND  IMPROVING  STRUCTURED  SIMULATION-BASED  TRAINING 


Introduction 

The  Close  Combat  Tactical  Trainer  (CCTT)  being  developed  by  the  U.  S.  Army 
Simulation,  Training,  and  Instrumentation  Command  (STRICOM)  will  support  maneuver 
training  for  fully-manned  platoon  and  company  units.  The  CCTT  also  will  support  battalion  task 
force  exercises,  with  leaders  in  manned  simulators  controlling  attached  semi-automated  force 
(SAP)  vehicles  that  fill  out  their  units.  The  CCTT  is  the  first  part  of  the  Combined  Arms 
Tactical  Trainer  (CATT)  system  providing  operational  training  for  total  combined  arms  forces  on 
a  virtual  battlefield.  The  U.  S.  Army  Research  Institute  (ARI)  is  developing  structured  training 
support  packages  (TSPs)  for  required  mission  and  task  training  using  the  CCTT.  Successful 
demonstration  of  structured  training  in  the  Virtual  Training  Program  (VTP)  with  Simulation 
Networking  (SIMNET)  established  a  model  (Burnside,  Leppert,  &  Myers,  1996)  for  similar 
CCTT  training. 

This  report  examines  needs  for  integrated  system  management  to  successfully  implement 
the  CCTT  and  its  TSPs.  While  CCTT  development  has  focused  on  meeting  simulation 
requirements,  and  TSP  development  has  focused  on  training  requirements,  total  system 
management  has  been  a  secondary  consideration.  Conceived  in  a  Total  Quality  Management 
(TQM)  framework,  this  report  proposes  processes  to  help  sustain  and  improve  the  training 
effectiveness  of  the  CCTT  throughout  system  life.  The  emphasis  is  on  providing  management 
feedback  by  continuously  monitoring  training  process  and  product  indicators,  and  on  establishing 
process  action  teams  to  identify  and  solve  system  problems. 

After  development  by  STRICOM,  the  fielded  CCTT  will  support  Active  Component 
units  with  company  sets  at  fixed  sites,  and  will  support  Reserve  Component  units  with  mobile 
platoon  sets.  Contractor  logistic  support  (CLS)  staff  will  maintain  and  operate  the  CCTT  system. 
CCTT  development  did  not  include  the  structured  training  program  as  an  integral  part  of  the 
system,  and  it  did  not  influence  system  design.  Thus,  the  current  CLS  contract  does  not 
explicitly  charter  nor  fully  resource  administration  and  implementation  of  this  training  program. 
Management  of  the  total  CCTT  training  system  necessarily  involves  shared  responsibility  and 
coordinated  action  among  the  training  units,  the  CLS  contractor,  and  the  installation  hosting  the 
units  and  the  CCTT  system.  Allocation  of  training  duties  and  responsibilities  were  considered  in 
the  design  of  CCTT  structured  training.  Additional  oversight  and  feedback  mechanisms  would 
assure  that  all  follow  recommended  procedures,  and  help  to  sustain  and  improve  the  quality  of 
training.  The  Total  Army  Quality  program  suggests  mechanisms  needed  to  establish  an  effective 
framework  for  managing  structured  training  programs. 

Army  experience  with  CCTT  structured  training  and  its  predecessor  VTP  program  should 
influence  future  simulation  standards.  The  approved  recommended  practice  standard  for 
distributed  interactive  simulation  (DIS)  (Institute  of  Electrical  and  Electronics  Engineers,  1996) 
only  treated  exercise  management  and  feedback  concerns  in  relation  to  single  exercises.  This 
standard  failed  to  consider  needs  associated  with  long  term  use  of  a  simulation  system  with  a 
standard  library  of  training  exercises.  The  CCTT  is  a  DIS  system  in  part,  but  also  is  compliant 
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with  standards  for  the  High  Level  Architecture  (HLA)  introduced  to  promote  simulation 
interoperability  and  software  reuse  (Under  Secretary  of  Defense  for  Acquisition  and  Technology, 
1995).  As  in  the  DIS  standard,  long-term  management  of  training  programs  is  an  issue  neglected 
in  the  HLA  federation  development  and  execution  process  model  (FEDEP)  (Defense  Modeling 
and  Simulation  Office,  1996).  The  FEDEP  model  is  being  revised  and  expanded  in  workshops 
conducted  by  the  Simulation  Interoperability  Standards  Organization  (Bouwens,  Freeman, 
Harkrider,  &  Zimmerman,  1997). 


Structured  Training 

Structured  training  provides  mission-based  and  task-focused  exercises  for  units  or  staff 
groups.  Deliberately  designed  in  the  exercises  are  specific  situations  and  events  that  assure 
appropriate  conditions  occur  for  practicing  performance  of  particular  tasks,  sub-tasks,  or  actions. 
Structured  training  derives  from  and  implements  the  Systems  Approach  to  Training  (SAT) 
prescribed  by  regulation  (U.  S.  Army  Training  and  Doctrine  Command,  1995;  Brown,  1993). 
Structured  training  may  be  developed  for  live,  virtual,  or  constructive  environments. 

Fundamental  Precepts 

The  VTP  developed  structured  training  exercises  designed  to  exploit  capabilities  of  the 
SIMNET  system  for  units,  and  Distributed  Janus  for  staffs.  The  U.  S.  Army  Research  Institute 
Armored  Forces  Research  Unit  (ARI-AFRU),  the  U.  S.  Army  Armor  Center  (USAARMC),  and 
the  Army  National  Guard  (ARNG)  jointly  directed  the  VTP  project.  Initially,  the  design  team 
adopted  several  training  principles  to  guide  the  development  (Campbell,  Campbell,  Sanders,  & 
Flynn,  1 995).  These  principles  remain  central  to  the  structured  approach  for  training  small  units. 
Later  projects  modified  and  expanded  them  for  larger  units  and  staffs  (Campbell  &  Deter,  1997). 

Scenario-Embedded.  A  logically  sequenced  scenario  for  a  complete  mission  with 
realistic  situations  and  conditions  places  exercises  in  meaningful  operational  context.  This 
context  should  aid  transfer  of  training. 

Execution  Focus.  Practice  emphasizes  actions  required  to  execute  the  mission.  This 
makes  best  use  of  simulation  capabilities  and  on-site  time.  The  unit  receives  mission  orders  and 
map  overlays  in  advance  of  scheduled  exercises  to  enable  it  to  conduct  plainning  and  preparation 
activities  off-site. 

Mission  Segments.  Exercise  duration  is  limited  by  executing  a  phase  of  the  total  mission 
lasting  one  hour  or  less  for  platoons  or  companies.  This  allows  frequent  after  action  reviews 
(AARs)  when  memory  for  events  remains  fresh.  The  end  of  one  phase  is  the  starting  point  of  the 
phase  in  the  next  exercise.  Higher-level  units  or  staffs  may  complete  longer  exercises  with  larger 
mission  segments  based  on  typical  pauses  in  the  mission  sequence. 

Task-Driven.  The  detailed  arrangement  of  scenario  events  deliberately  elicit  and 
constrain  execution  of  specific  tasks  that  form  the  training  objectives,  thereby  enabling 
observation  and  evaluation  of  performance  based  on  task  standards.  The  same  task  objectives  set 
the  framework  for  an  AAR  designed  to  provide  carefully  structured  performance  feedback. 
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Compressed  Time.  Trainers  follow  a  set  agenda  to  deliver  as  much  training  as  possible  in 
the  available  time.  Preview,  execution,  and  AAR  activities  total  about  two  hours  for  platoon  or 
company  exercises.  In  execution,  tasks  follow  each  other  in  close  sequence  with  minimal  slack 
time,  and  with  few  events  and  actions  unrelated  to  the  training  objectives.  Efficient  training  is  a 
priority  for  the  VTP  because  of  limited  time  available  to  ARNG  and  Reserve  units. 

Fully  Supported.  TSPs  provide  unit  preparation  guidance,  and  detailed  event  guides  and 
other  materials  for  observer/controllers  (0/Cs)  and  simulator  operators  who  conduct  and  assist 
the  training.  These  trainers  provide  “turn-key”  operation  of  the  simulation  and  training  system. 
Unit  personnel  cannot  be  expected  to  acquire  technical  knowledge  of  system  capabilities  and 
operations  to  the  degree  necessary  for  effective  simulation-based  training.  Commanders  and 
staffs  also  should  not  expend  precious  time  repeatedly  creating  exercises  that  “reinvent  the 
wheel”  before  training  their  units  in  simulators. 

Standardized  Library.  A  sufficient  number  of  TSPs  are  made  available  to  support 
repeated  practice  of  basic  skills  needed  to  execute  common  tasks  in  the  Mission  Essential  Task 
List  (METL)  for  a  particular  type  of  unit.  The  library  provides  gradually  increasing  levels  of 
difficulty,  and  unit  progression  in  the  sequence  should  be  performance-based.  After  mastering 
basic  skills  with  standard  TSPs,  new  exercise  variations  and  new  missions  can  provide  additional 
challenges. 

Previous  Developments 

The  VTP  project  (Hoffman,  Graves,  Roger,  Flynn,  &  Sever,  1995)  created  a  training 
library  based  on  two  missions  for  mechanized  infantry  and  tank  units  with  over  100  structured 
platoon,  company,  battalion,  and  battalion  staff-level  exercises.  This  library  also  included 
exercises  for  scout  platoons  and  cavalry  troops.  Shlechter,  Bessemer,  Nesselroade,  &  Anthony 
(1995)  report  evidence  for  training  effectiveness  obtained  during  initial  exercise  trials.  Strong 
evidence  for  program  success  is  the  increased  demand  for  VTP  training  over  several  years  as 
sho-wn  in  Table  1  (Russell,  1996).  After  focusing  on  platoon-level  training  in  the  early  years  of 
the  VTP,  training  at  the  more  advanced  company-level  increased  in  1997. 

Table  1 

Number  of  Units  Served  Yearly  by  the  VTP 


Year 

Unit  Level 

FY  94 

FY  95 

FY  96 

FY  97' 

Platoon 

107 

265 

367 

169 

Company 

23 

55 

41 

121 

Battalion 

14 

28 

36 

13 

'  Numbers  provided  by  the  Armored  Warfighting  Training  Directorate  0/C  Team. 


Follow-up  questionnaires  and  interviews  with  departing  members  of  the  initial  VTP  0/C 
team  (Shlechter,  Kraemer,  Bessemer,  &  Burnside,  1996)  showed  high  consensus  that  VTP 
training  substantially  improved  the  tactical  performance  of  both  leaders  and  units.  The  0/Cs 
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observed  that  leaders  gained  in  command  and  control  skills,  situational  awareness,  and  apparent 
self-confidence.  Units  gained  in  reporting,  actions  on  contact,  formation  execution,  teamwork, 
and  cohesive  action.  However,  The  0/Cs  pointed  out  several  aspects  of  the  materials  and 
procedures  that  needed  fine-tuning.  The  0/Cs  also  noted  that  units  require  better  advance 
preparation  before  their  VTP  training. 

Subsequent  projects  using  the  same  structured  approach  have  expanded  the  VTP  to 
brigade  staffs  (Koger,  Long,  Britt,  Sanders,  Broadwater  &  Brewer,  1996),  a  third  mission 
(Graves  &  Myers,  1997),  and  leaders  of  combat  support  (CS)  and  combat  service  support  (CSS) 
platoons  (Hoffman,  in  press).  BDM  Federal  led  the  contractor  team,  with  Human  Resources 
Research  Organization  (HumRRO)  and  Litton-PRC,  to  develop  the  original  VTP  and  later 
expansions,  monitored  by  ARI-AFRU. 

Another  recent  project  developed  structured  brigade  staff  exercises  using  the 
Brigade/Battalion  Simulation  (BBS)  (Graves,  Campbell,  &  Deter,  in  press).  This  project  also 
developed  brigade  staff  section  exercises  for  “live”  tactical  operations  centers  (TOCs),  and  for 
BBS  and  Janus  environments.  For  this  work,  HumRRO  led  the  contractor  team,  with  BDM 
Federal,  Hughes  Training,  and  Litton-PRC,  and  monitored  by  ARI-AFRU.  Campbell,  Deter,  & 
Quinkert  (1997)  describe  the  extension  of  the  development  methodology  for  structured  training 
to  brigade  level. 

CCTT  Structured  Training 

A  contractor  team  developed  an  initial  limited  set  of  40  structured  TSPs  for  the  CCTT 
(STRUCCTT  Team,  1997).  Senior  Army  personnel  selected  the  exercises  in  this  set  to  parallel 
some  of  those  available  in  the  VTP  for  tank  and  mechanized  infantry  platoons  and  company 
teams  performing  three  missions:  movement  to  contact,  deliberate  attack,  and  defense  in  sector. 
Besides  the  initial  TSPs,  the  team  designed  complete  exercise  libraries  for  these  types  of  units 
and  missions.  In  addition,  the  team  did  one  battalion  task  force  TSP  for  a  movement  to  contact 
mission.  The  Operational  Test  &  Evaluation  Command  (OPTEC)  funded  the  CCTT  TSP 
development  to  support  acquisition  testing  and  evaluation  of  the  CCTT  system.  The  contract 
titled  “Structured  Training  for  Units  in  the  Close  Combat  Tactical  Trainer”  (STRUCCTT)  was 
monitored  by  ARI-AFRU.  HumRRO  led  the  contractor  team  with  BDM  Federal,  Hughes 
Training,  and  Litton-PRC.  A  follow-on  effort  is  developing  exercises  for  the  heavy  cavalry  troop 
and  scout  platoon,  and  a  task  force  exercise  for  a  defense  in  sector  mission.  Work  has  also  begun 
to  design  a  training  program  and  TSPs  for  the  tactics,  techniques,  and  procedures  (TTP) 
employed  in  digital  operations  with  Ml  A2  tanks  and  M2A3  infantry  vehicles. 

New  CCTT  simulation  capabilities  unavailable  in  SIMNET  substantially  enhance  the 
CCTT  TSPs  compared  to  their  VTP  counterparts.  For  example,  CCTT  allows  variation  in 
visibility  conditions.  Advanced  Semi-Automated  Forces  (SAF),  CS,  and  CSS  workstations,  and 
dismounted  infantry  squad  stations  operated  by  squad  leaders  provide  many  more  capabilities. 
The  CCTT  TSPs  include  as  many  new  conditions  as  possible  in  at  least  one  exercise  to  enable 
comprehensive  testing.  The  TSP  content  and  techniques  have  been  reviewed  by  the  Armor  and 
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Infantry  Schools  and  approved  by  the  Training  and  Doctrine  Command  (TRADOC)  System 
Manager  for  CATT  (TSM-CATT). 

In  1997,  first  use  of  the  TSPs  occurred  during  CCTT  system  integration  testing  and  a 
pilot  test.  After  adjustments  for  CCTT  software  changes,  the  STRUCCT  team  completed  a 
formative  evaluation  of  the  TSPs  during  a  Limited  User  Test  (LUT)  at  Fort  Hood,  TX.  The 
refined  TSPs  are  now  being  used  in  the  CCTT  Initial  Operational  Test  &  Evaluation  in  1998. 

Army  Management 

Serious  efforts  to  spread  the  philosophy  and  methods  of  Total  Quality  Management 
(TQM)  throughout  the  federal  government  began  by  establishing  the  Federal  Quality  Institute  in 
1988.  By  1992,  over  2/3  of  federal  agencies  surveyed  reported  using  TQM  in  some  way  (Shoop, 

1 993).  In  that  same  year,  the  U.  S.  Army  formally  adopted  TQM  as  part  of  its  management 
philosophy  (Department  of  the  Army,  1992).  Under  the  banner  of  “Total  Army  Quality”  (TAQ), 
offices  opened  at  all  major  commands  and  installations.  The  TAQ  offices  train  Army  leaders  and 
the  military  and  civilian  workforce,  giving  them  both  concepts  and  tools  needed  to  change  Army 
management  practices.  To  quote  General  Sullivan  (Office  of  Total  Army  Quality,  1996),  who 
was  Army  Chief  of  Staff  in  1992,  “The  implementation  of  the  Army’s  Total  Army  Quality 
philosophy  is  not  optional.”  However,  he  went  on  to  state  that  leaders  must  tailor  TAQ  to  fit 
their  own  organizations.  The  TAQ  approach  has  been  largely  non-prescriptive,  encouraging 
demonstration  projects  to  improve  processes  and  results. 

Performance  Improvement  Criteria 

As  an  inducement  to  implement  TAQ  management,  the  Army  conducts  a  competition 
with  wirming  installations  receiving  substantial  monetary  awards.  Organizations  submit  entries 
describing  practices  in  six  areas  corresponding  to  the  Malcolm  Baldrige  categories  in  civilian 
TQM  competition,  plus  one  category  added  by  the  Army  (Office  of  the  Army  Chief  of  Staff, 
1997): 

•  Leadership 

•  Strategic  Planning 

•  Customer  and  Market  Focus 

•  Information  and  Analysis  (added  category) 

•  Human  Resource  Development  and  Management 

•  Process  Management 

•  Business  Results 

The  seven  categories  subsume  20  items,  with  detailed  criteria  for  each  item.  Only  the 
Human  Resource  Development  and  Management  category  relates  to  training.  It  includes  criteria 
for  employee  education,  training,  and  development  in  general  terms,  but  does  not  address  soldier 
skills  or  training  for  combat.  The  imperative  to  apply  TAQ  to  military  training  is  an  inference 
from  unit  missions. 
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TAQ  Infrastructure 


In  a  nutshell,  the  Army’s  management  philosophy  is  to  “Do  the  right  things,  the  right 
way,  for  the  right  reasons,  and  constantly  strive  for  improvement”  (Department  of  the  Army, 
1992).  One  key  to  this  philosophy  is  leadership  as  the  first  Baldrige  category  suggests.  For 
success,  TQM  requires  managers  to  have  constancy  of  purpose,  shared  strategic  vision, 
deployment  of  that  vision  in  goals  and  action  plans,  determination  to  provide  the  necessary 
resources,  and  willingness  to  fully  empower  subordinates  to  develop  and  adopt  process  changes. 
The  senior  commander  and  subordinates  must  be  entirely  committed  to  quality  improvement 
principles  and  the  worth  of  the  effort  required.  Leader  involvement  in  working  groups  organized 
to  direct  or  execute  TAQ  projects  reflects  this  commitment. 

As  one  example.  Fort  Knox  established  a  three-tiered  system  of  TAQ  working  groups:  an 
Executive  Steering  Committee  (ESC),  three  Quality  Workforce  Boards  (QWBs),  and  Process 
Action  Teams  (PATs)  as  needed  for  specific  projects  (Office  of  Total  Army  Quality,  1996).  The 
ESC,  composed  of  senior  leaders,  is  responsible  for  TAQ  policy,  developing  the  USAARMC 
vision,  setting  goals  and  priorities,  directing  actions,  and  monitoring  progress.  QWBs  include 
organizational  leaders  responsible  for  TAQ  actions  in  the  areas  of  base  operations,  training,  and 
futures.  The  QWBs  identify  processes  in  their  assigned  areas,  set  action  priorities,  and  establish 
PATs  for  specific  projects.  QWBs  then  review  results,  and  approve  and  lead  changes  in  their 
organizations. 

PATs  apply  TAQ  methods  to  selected  work  processes  in  the  scope  of  their  charters.  They 
identify  and  solve  problems,  and  implement  effective  changes.  PATs  are  cross-functional  with 
members  assigned  as  needed  to  work  on  a  project.  A  PAT  will  include  representatives  of 
“owners”  (organization  leaders)  of  chosen  work  processes,  expert  personnel  directly  involved 
and  in  these  processes,  experts  on  related  processes  (delegated  by  suppliers  or  customers  that 
interact  with  the  primary  organization).  The  PAT  also  may  include  technical  advisors  or 
consultants  for  specific  issues  or  methods.  A  facilitator,  expert  in  TAQ  methods,  is  assigned  to 
train,  coach,  and  assist  the  PAT,  and  uses  team-building  techniques  to  improve  its  work. 

TAQ  in  Practice 

The  core  TQM  instrument  of  change  and  continuous  improvement  is  a  scientific  problem¬ 
solving  approach  based  on  facts  about  both  process  and  results.  Instead  of  managing  by  results, 
quality  leadership  manages  processes  to  produce  results  (Sholtes,  1988).  This  approach  employs 
the  Plan-Do-Check- Act  (PDCA)  or  “Shewart”  cycle  described  by  Deming  (1986)  and  Walton 
(1986).  Continuous  improvement  results  from  cycle  repetition.  Sources  often  expand  and 
elaborate  The  PDCA  cycle  in  different  ways,  but  the  TAQ  Office  at  Fort  Knox  (Office  of  Total 
Army  Quality,  1996)  teaches  six  steps: 
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•  Define  the  Problem  (Plan) 

•  Identify  Possible  Causes  (Plan) 

•  Evaluate  Possible  Causes  (Plan) 

•  Make  a  Change  (Do) 

•  Test  the  Change  (Check) 

•  Take  Permanent  Action  (Act) 

A  PAT  uses  specific  methods,  techniques,  and  tools  to  perform  each  step  (GOAL/QPC, 
1988;  Sholtes,  1988).  To  summarize  briefly,  the  PAT  carries  out  the  Plan  steps  to  develop  a 
detailed  understanding  of  a  work  process  producing  a  product  or  delivering  a  service.  This 
understanding  includes  the  customer’s  criteria  of  quality  (value)  for  the  process  output,  the 
sequence  of  actions  and  decisions  leading  to  the  output,  workforce  roles,  inputs  to  the  process, 
and  known  problems.  Based  on  the  process  description,  the  PAT  defines  performance  measures, 
collects  and  analyzes  data  to  examine  the  state  of  the  process,  and  identifies  or  confirms 
problems.  The  PAT  also  speculates  about  cause-effect  relations,  and  collects  data  to  evaluate 
cause-effect  hypotheses. 

At  the  next  step  (Do),  the  PAT  develops  and  tries  out  a  process  change  on  a  limited  scale. 
The  effect  of  the  change  is  measured  (Check)  to  determine  if  the  result  meets  expectations.  The 
PAT  may  compare  results  before  and  after  the  change,  or  test  the  effect  by  formal  controlled 
experiment.  If  results  confirm  the  benefit  of  the  change,  the  last  step  makes  it  permanent  (Act). 

The  USAARMC  has  completed  several  successful  TAQ  projects,  mainly  related  to 
processes  in  base  operations.  The  Armor  School  formed  just  one  PAT  to  address  an  issue 
directly  related  to  training.  This  PAT  sought  to  improve  methods  used  to  survey  customer 
organizations  about  the  quality  of  assigned  course  graduates.  The  Armor  School  did  not  fully 
test  or  implement  planned  changes  in  that  case.  Organizations  that  manage  the  VTP  program  or 
other  simulation  training  have  not  conducted  TAQ  projects. 

CCTT  Training  Management 

In  theory,  U.  S.  Army  units  manage  training  in  a  repeated  cycle:  assessment  of  training 
needs,  planning  and  scheduling,  executing  training,  evaluation,  and  reassessment  (Department  of 
the  Army,  1990).  The  structured  training  processes  and  the  responsibilities  assigned  were 
designed  to  be  compatible  with  this  cycle.  Structured  training  supports  the  general  goals  of  using 
the  CCTT  to  prepare  for  subsequent  unit  training  in  field  exercises,  and  to  sustain  unit 
proficiency  after  field  training.  The  processes  also  fit  within  the  constraints  imposed  by  the 
simulation  equipment  provided  at  CCTT  facilities. 

CCTT  Components  and  Operation 

A  typical  CCTT  site  will  have  two  company  sets  of  maimed  simulator  modules,  including 
14  Ml  A 1  or  Ml  A2  Abrams  tanks,  and  14  M2A2  or  M2A3  Bradley  infantry  fighting  vehicles. 
Two  dismounted  infantry  modules  are  available  for  use  with  the  infantry  vehicles.  The  site 


7 


provides  one  M981  Fire  Support  Team  Vehicle,  as  well  as  one  Ml  13  Armored  Personnel  Carrier 
and  one  High  Mobility  Multi-Purpose  Wheeled  Vehicle  for  various  roles. 

A  Master  Control  Console  (MCC)  and  five  AAR  workstations  provide  system 
initialization  and  control.  The  AAR  stations  independently  can  control  several  simultaneous 
exercises.  Supporting  these  exercises,  ten  SAF  workstations  control  Blue  Force  (BLUFOR)  or 
Opposing  Force  (OPFOR)  simulations  of  unmanned  vehicles  and  units  of  many  possible  types, 
all  operating  in  a  tactically  realistic  manner.  Eight  Unit  Support  Workstations  (USW)  are 
available  to  simulate  a  number  of  CS  and  CSS  functions: 

•  Mortar  Fire  Direction  Center  (FDC) 

•  Field  Artillery  Battalion  TOC  (FABTOC) 

•  Fire  Support  Element  (FSE) 

•  Combat  Engineer  Support  (CES) 

•  Tactical  Air  Control  Party  (TACP) 

•  Combat  Trains  Command  Post  (CTCP) 

•  Unit  Maintenance  Collection  Point  (UMCP) 

•  Higher  Headquarters  Support  (HHS) 

Full-time  CLS  staff  operate  the  MCC,  SAF,  and  AAR  workstations.  However,  unlike  the 
VTP  in  SIMNET,  the  CCTT  by  design  has  no  dedicated  full-time  trainers.  The  CCTT  device 
requirement  specified  operation  without  added  Army  personnel  spaces  (U.  S.  Army  Training 
Support  Center,  1998).  To  conduct  training,  the  unit’s  higher  command  must  assign  one  or  more 
0/Cs  and  several  USW  operators  (usually  at  least  three),  or  the  unit  must  recruit  personnel  from 
sister  and  supporting  units.  Sites  also  have  personal  computers  with  Interactive  Courseware 
(ICW)  programs  that  provide  self-paced  workstation  operator  training  (also  called  EDUCATT). 

CCTT  Structured  Training  Process 

Figure  1  presents  an  outline  of  the  structured  training  process  used  with  the  CCTT.  The 
pre-  and  post-exercise  phases  include  CCTT-related  activities  done  off-site  before  and  after  the 
unit  trains  at  the  CCTT  site.  On-site  activities  include  some  general  preparations  for  CCTT 
operations,  leading  to  unit  training  in  one  or  more  specific  exercises.  Each  exercise  has  preview, 
execution,  and  AAR  phases. 

In  the  pre-exercise  phase  the  unit  commander  and  staff  assess  training  needs  and  elect  or 
receive  direction  to  include  the  CCTT  in  their  training  plans.  They  contact  the  CCTT  Site 
Manager  who  provides  TSP  materials  and  schedule  information.  After  reviewing  materials  and 
selecting  exercises,  they  schedule  training,  and  can  request  exercise  modifications.  While  the 
CLS  operators  can  make  some  minor  changes  when  an  exercise  is  initialized,  most  substantial 
changes  in  initialization  files  must  be  made  well  in  advance  of  the  training  date.  Large 
modifications  of  the  exercises  also  require  revision  of  the  TSP  materials  to  make  the  0/C  and 
USW  operator  guides  consistent  with  the  actual  exercise  situation.  The  unit  staff  must  also 
coordinate  and  schedule  0/Cs  and  USW  operator  support,  schedule  on-site  ICW  (EDUCATT) 
training  for  them,  and  distribute  TSP  guidance  materials. 
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CCTT  Training  Phases 


Figure  1.  Phases  of  structured  training  process. 

Nejir  the  training  date,  the  unit  commander  and  staff  issue  orders  and  map  overlays  for  all 
planned  exercises,  conduct  troop-leading  procedures,  and  supervise  unit  rehearsals.  The  0/Cs 
and  workstation  operators  must  study  their  TSP  exercise  guidance  materials,  and  participate  in 
unit  rehearsals.  Meanwhile,  the  Site  Manager  assigns  CLS  operators  and  trains  them  when 
needed  to  maintain  proficiency.  The  CLS  operators  review  TSP  guidance  materials  in  advance 
for  the  scheduled  exercises,  and  make  requested  changes  in  exercise  files. 

Preparation  often  begins  during  the  week  before  the  scheduled  training.  The  0/Cs  and 
USW  operators  visit  the  site  to  take  the  EDUCATT ICW  training  for  their  positions.  CLS 
operators  then  conduct  a  workstation  orientation  exercise  with  the  0/Cs  and  USW  operators  to 
apply  and  practice  what  they  learned  in  ICW  training.  When  the  unit  arrives  for  training,  site 
staff  conducts  unit  preparation  activities,  presenting  an  introductory  briefing  to  provide  an 
overview  of  CCTT  training  and  site  ground-rules,  and  leading  a  site  tour  showing  the  facility 
layout.  Vehicle  crews  then  participate  in  an  orientation  exercise  enabling  them  to  adjust  to  the 
operation  of  manned  simulator  module  displays,  controls,  and  communication  equipment,  and  to 
navigation  on  simulated  terrain.  The  exercise  also  provides  experience  with  the  visual 
appearance  of  vehicles,  cultural  features,  and  weapon  effects  represented  in  the  CCTT 
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simulation.  Before  each  training  exercise,  CLS  operators  check  the  CCTT  system,  and  activate 
appropriate  initialization  files. 

Training  exercises  begin  with  a  preview.  The  0/C  summarizes  the  mission  and  situation 
in  the  context  of  higher-level  orders,  and  reviews  tasks  that  the  unit  will  perform.  The  unit 
commander  briefs  back  his  intent,  operations  order,  and  plan  for  executing  the  mission.  Crews 
then  occupy  their  vehicle  modules  and  conduct  pre-operations  checks.  At  the  same  time,  the 
commander  uses  the  terrain  display  at  the  AAR  station,  assisted  by  the  AAR  operator,  to  conduct 
a  reconnaissance  of  the  area  of  operation  with  his  subordinate  leaders. 

During  exercise  execution,  the  0/C,  AAR  and  SAP  operators,  and  USW  operators  all 
follow  a  detailed  event  guide  showing  when  to  initiate  various  actions  or  communications.  The 
event  guide  helps  to  create  the  tactical  environment  requiring  performance  of  tasks  trained  in  that 
exercise.  At  the  AAR  station,  the  0/C  listens  to  radio  transmissions  and  sends  scripted  messages 
to  play  the  role  of  the  superior  commander  and  other  elements  with  which  the  unit  interacts  on 
the  radio.  He  regulates  the  timing  of  actions  by  the  supporting  SAP  and  USW  operators, 
observes  unit  performance  on  the  visual  displays,  and  notes  items  for  later  reference  in  the  AAR. 
The  AAR  operator  positions  the  view  of  the  battlefield  so  the  0/C  can  observe  from  an 
advantageous  point.  SAP  operators  control  the  actions  of  OPPOR  units  and  friendly  BLUPOR 
units  following  0/C  directions  and  the  event  guide.  USW  operators  communicate  with  the  0/C 
and  unit  as  they  would  on  an  actual  battlefield  in  that  situation,  and  execute  the  CS  and  CSS 
functions  called  for  by  the  event  guide. 

In  the  AAR,  the  0/C  follows  an  outline  focused  on  tasks  performed  in  the  exercise.  He 
facilitates  the  unit’s  discussion  of  what  happened,  why  it  happened,  and  how  to  improve  future 
performance.  SAP  and  USW  operators  may  participate  to  clarify  events  from  their  perspective. 
The  O/C  leads  the  unit  to  self-assess  some  aspects  of  performance  that  the  unit  should  sustain  or 
improve  in  the  future.  The  AAR  operator  assists  the  O/C  by  replaying  selected  exercise 
segments,  including  radio  traffic,  showing  important  elements  of  performance  for  discussion. 

After  completing  CCTT  training,  the  O/C  prepares  a  task-based  Post-Exercise  Report  on 
unit  performance.  The  unit  commander  reviews  and  distributes  the  report  to  his  staff  and  other 
subordinates.  Ideally,  they  all  should  use  the  report  to  reassess  the  unit’s  training  status,  and  to 
update  and  revise  training  plans  based  on  their  assessment. 

Threats  to  Effectiveness 

The  CCTT  structured  training  process  is  rather  complex.  Several  factors  can  degrade  the 
effectiveness  of  the  training  delivered  by  the  program.  In  the  beginning,  initial  unfamiliarity, 
divided  responsibilities,  the  lack  of  dedicated  trainers,  and  time  limitations  may  impede  full 
implementation  of  the  designed  training  process.  The  novelty  of  the  CCTT  and  the  structured 
TSPs,  and  the  sheer  bulk  of  the  new  guidance  materials  require  repeated  study  to  absorb  all  the 
details.  The  division  of  responsibility  and  coordination  required  among  the  installation 
command,  units,  and  site  makes  it  easy  to  overlook  important  steps  or  to  leave  them  unfinished. 
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For  example,  if  distribution  of  TSP  materials  is  late,  unit  leaders,  0/Cs,  and  USW  operators  may 
not  have  an  opportunity  to  study  the  materials  in  detail.  Furthermore,  commanders  may  not 
identify  exercise  modifications  that  would  better  meet  the  unit’s  training  needs,  or  may  not  be 
able  to  request  changes  with  sufficient  lead-time. 

Without  a  cadre  of  dedicated  trainers,  the  TSP  developers  initially  can  only  train  the  CLS 
operators.  These  operators  must  then  continually  train  new  0/Cs  and  USW  operators.  The 
constant  turnover  of  0/Cs  and  USW  operators  necessarily  limit  their  mastery  of  the  CCTT 
equipment  and  the  guidance  materials  for  the  training  exercises.  Finally,  units  currently  have  full 
training  calendars  and  many  other  obligations,  and  will  find  it  difficult  to  allot  much  time  to 
CCTT  training.  Advance  troop  leading  procedures  and  unit  rehearsals  needed  to  prepare  for 
training  may  be  minimal  or  even  omitted.  Last  minute  preparation  on-site  may  reduce  the  time 
used  in  CCTT  training.  Time  also  is  limited  for  critical  personnel  to  train  and  serve  as  0/Cs  or 
USW  operators.  The  result  can  be  under-use  of  the  CCTT,  or  delivery  of  poor  training  by 
trainers  lacking  sufficient  preparation. 

Long  term,  many  of  the  initial  CCTT  problems  may  be  reduced,  but  not  entirely 
eliminated,  as  experience  using  the  CCTT  spreads,  and  units  return  for  repeated  training. 
However,  other  factors  will  remain  in  play  throughout  the  life  of  the  CCTT  system.  As 
technology  advances,  the  CCTT  will  have  frequent  hardware  and  software  upgrades.  During 
earlier  product  integration  testing,  such  changes  often  required  revision  of  exercise  initialization 
files  and  TSP  materials.  Similar  revisions  will  be  necessary  as  CCTT  continues  to  expand  and 
improve.  The  CCTT  Training  Device  Requirement  (U.  S.  Army  Training  Support  Center,  1998) 
lists  many  preplanned  product  improvements  in  a  variety  of  areas.  Furthermore,  as  the  Army 
moves  toward  digital  operations,  CCTT  training  will  have  to  incorporate  a  constant  flow  of  new 
doctrine,  missions,  METL  tasks,  and  TTPs.  To  meet  these  new  training  needs,  existing  exercises 
will  require  modification  and  new  exercises  developed  to  expand  the  TSP  library.  Wilkinson 
(1997)  suggested  that  intelligent  software  is  needed  to  assist  CCTT  users  in  selecting,  modifying, 
or  creating  new  exercises,  and  such  software  now  is  being  developed  (U.S.  Army  Research 
Institute  for  the  Behavioral  and  Social  Sciences,  1997). 

The  challenges  are  to  assure;  (a)  that  the  CCTT  training  process  is  implemented  as 
designed,  (b)  that  a  high  level  of  training  effectiveness  is  maintained  with  consistency,  and  (c) 
that  the  total  training  system  can  adapt  to  a  changing  environment.  To  meet  these  goals  while 
overcoming  both  short-  and  long-term  potential  problems,  TQM  mechanisms  can  provide  the 
corrective  actions  and  continuous  improvement  needed  to  maintain  and  strengthen  the  total 
CCTT  system.  With  such  a  training  system  that  is  continually  evolving,  any  short-term  test  or 
evaluation  can  only  provide  a  snapshot  assessment  of  its  current  state  at  one  point  in  time. 

System  managers  must  monitor  and  evaluate  system  processes  and  products  continuously 
throughout  its  life  cycle  to  sustain  the  expected  return  on  the  Army’s  investment. 

Applying  TAQ  Methods  To  The  CCTT 

The  specific  procedures  defined  in  the  CCTT  structured  training  process  facilitate 
application  of  TAQ  methods.  TAQ  management  projects  can  therefore  start  from  an  established 
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training  process  model.  This  model  provides  a  baseline  condition  and  point  of  departure  for 
developing  and  evaluating  changes. 

Organizing  for  TAQ  Management 

To  provide  command  emphasis  and  leadership  for  CCTT  TAQ  projects,  the  installation 
commander  should  establish  a  steering  committee  with  the  same  functions  as  the  Fort  Knox  ESC. 
Members  should  include  the  commander  himself,  senior  leaders  or  representatives  of  major  local 
commands  and  any  other  organizations  that  relate  to  CCTT  operation  and  training,  the  CCTT 
Site  Manager,  and  the  head  of  the  local  TAQ  office.  Members  also  should  come  from  agencies 
responsible  for  funding  the  CCTT  system  and  facility.  This  TAQ  effort  should  not  require  an 
intermediate  quality  board  unless  the  commander  wants  to  expand  the  projects  to  include  all  local 
training  processes.  The  ESC  should  set  initial  goals,  and  establish  and  charter  a  PAT.  Possible 
goals  for  an  initial  effort  are  (a)  to  describe  the  training  process,  (b)  to  develop  process 
indicators,  and  (c)  to  identify  major  problems  for  investigation. 

The  PAT  chair  should  be  the  supervisor  of  the  CLS  operator  staff  Members  should 
include  senior  AAR  and  SAF  operators,  armor  and  infantry  unit  leaders  at  platoon  and  company 
levels  with  CCTT  training  experience,  and  CCTT  0/Cs  with  experience  in  platoon  and  company 
training.  Various  USW  operators  should  participate  when  their  functions  are  under 
consideration.  The  local  TAQ  office  should  provide  a  trained  facilitator. 

Identifying  Problems 

The  PAT  should  begin  by  preparing  a  series  of  flowcharts  (Walton,  1986;  GOAL/QPC, 
1988)  showing  the  actual  step-by-step  sequence  of  actions  and  decisions  that  complete  phases  of 
the  training  process.  Branches  show  alternatives  when  ways  of  performing  a  process  segment 
vary.  The  PAT  should  compare  these  charts  to  the  “ideal”  process  designed  by  the  TSP 
developers  (STRUCCTT  Team,  1997)  to  find  differences.  Examination  of  the  differences  must 
determine  if  they  are  possible  problem  sources,  or  process  improvements.  The  PAT  looks  for 
gaps,  loops,  conflicts,  or  complexities  that  may  lead  to  proeess  mistakes,  breakdowns,  delays, 
inefficiencies,  and  variation.  Brainstorming,  multi-voting,  or  nominal  group  techniques  (Sholtes, 
1988)  may  be  used  to  nominate,  select,  and  set  priorities  for  further  work. 

Further  detail  on  the  allocation  of  responsibilities  may  be  described  by  deployment  charts 
(Sholtes,  1988)  that  show  “suppliers”  providing  input  and  “customers”  receiving  output  of  each 
step,  and  “owners”  who  perform  each  step.  These  charts  identify  who  must  make  any  needed 
changes  in  the  process. 

The  PAT  should  also  perform  cause-effect  analysis  (Sholtes,  1988;  GOAL/QPC,  1988) 
on  process  stages  or  functions.  This  method  focuses  on  the  result  or  outcome,  defining  desired 
and  undesired  characteristics  for  the  output  of  a  stage  or  function.  Examining  the  flowchart,  the 
PAT  seeks  to  trace  back  to  root  causes  all  factors  that  influence  the  output.  A  causal  tree 
diagram  (the  Ishikawa  “fishbone”  diagram  in  Figure  2)  is  used  to  record  the  analysis  with  causal 
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CAUSE-EFFECT  DIAGRAM 
(FISHBONE) 


Figure  2.  Tree  diagram  with  major  causal  categories. 

chains  drawn  as  leaves  and  branches  from  main  limbs  showing  major  categories  of  causes. 

These  diagrams  identify  aspects  of  the  process  providing  concepts  for  measurement. 

Developing  Quality  Indicators 

For  parts  of  the  process  selected  for  priority  attention,  the  PAT  must  develop  measures  to 
assess  how  the  sub-process  is  working  in  the  current  system.  In  addition,  they  should  develop 
criteria  of  quality  for  the  product  or  output  of  the  total  process.  The  flow  charts  and  cause-effect 
diagrams  lay  out  a  myriad  of  possible  concepts  for  measurement.  The  PAT  selects  indicators 
that  have  the  greatest  impact  on  the  ultimate  criteria  of  quality,  that  are  relatively  objective,  and 
that  are  feasible  and  not  too  costly.  Advice  from  an  experienced  quality  management  consultant 
may  be  useful  in  selecting  indicators  and  setting  up  an  efficient  data  collection  system.  A  later 
section  of  this  report  presents  some  candidate  indicators  for  CCTT  structured  training. 

Indicators  must  be  operationally  defined  to  specify  who  must  observe  and  record  what 
events  or  variables,  at  what  locations,  and  under  what  circumstances.  Definitions  must  include 
the  units  and  scales  of  measurement  for  each  indicator.  Checklists,  forms,  and  surveys  organize 
the  selected  measures  for  data  collection.  A  detailed  plan  should  specify  the  data  collection 
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process  and  procedures  for  reproducing,  distributing,  collecting,  and  processing  completed 
forms.  Some  clerical  staff  may  be  required  to  monitor  data  collection,  audit  records,  and  manage 
databases.  Data  collection  must  be  integrated  with  other  activities  to  avoid  interfering  with  the 
process  under  observation,  to  reduce  the  data  recording  burden  combined  with  training  duties, 
and  to  minimize  added  costs  for  site  operation. 

All  personnel  involved  in  the  CCTT  training  process  will  contribute  data  under  some 
circumstances.  Site  managers  can  report  administrative  data.  Unit  leaders  and  0/Cs  can  supply 
information  on  pre-  and  post-exercise  activities  by  using  checklists  provided  with  the  Unit 
Training  Guide  and  Post-Exercise  Report  forms.  Units  can  fill  out  customer  satisfaction  surveys 
before  and  after  on-site  training.  The  0/C,  AAR  and  SAP  operators,  and  USW  operators  can 
provide  training  data  using  the  event  guide,  data-entry  windows  added  to  workstations,  and  post¬ 
exercise  forms.  Operators  can  accumulate  automated  performance  data  records  from  the  CCTT 
system.  The  ICW  computers  also  can  collect  automated  data  from  EDUCATT  training. 

Monitoring  Processes 

Collection  of  indicator  data  should  continue  over  a  long  time  for  all  units  using  the  CCTT 
TSPs.  Data  analysis  should  use  standard  graphic  methods  that  have  proven  their  value  in  quality 
projects.  For  example,  Pareto  charts  show  ordered  relative  frequencies  of  categories  to  highlight 
the  most  important  problems  or  causes  identified  in  a  set  of  alternatives. 

Control  charts  with  qualitative  or  quantitative  data  show  process  stability  or  trends  over 
time.  Such  charts  also  show  how  much  variation  normally  occurs  in  process  measures.  Control 
charts  help  to  identify  changes,  and  extreme  values  traceable  to  some  “special”  cause  (Deming, 
1986).  Early  improvement  efforts  seek  to  prevent  trends  and  special  causes,  leading  to  a  stable 
process  with  variation  resulting  only  from  “common”  causes  inherent  in  the  process.  A  stable 
process  establishes  a  baseline  for  evaluating  the  effect  of  planned  process  improvements. 

Stratification  (Sholtes,  1988),  regression,  and  structural  equation  analyses  also  can  be 
used  to  examine  hypotheses  about  cause-effect  relationships  contributing  to  common  variation. 
Many  commercial  and  statistical  software  packages  are  available  to  do  these  analyses. 

Developing  Changes 

As  the  PAT  identifies  problems,  it  should  focus  first  on  changes  producing  short-term, 
big  wins  to  increase  the  impetus  for  TAQ  management  projects.  Sholtes  (1988)  presents  1 1 
strategies  for  change,  but  the  basic  approach  remains  constant.  The  PAT  must  define  the  reason 
for  change  and  the  goal  to  be  met  in  terms  of  measurable  criteria,  then  design  alternatives  and 
examine  their  advantages  and  disadvantages.  Changes  also  must  meet  situational  constraints  and 
avoid  unintended  consequences.  The  PAT  must  plan  changes,  have  them  approved  by  the  ESC, 
and  give  training  to  those  affected.  The  organization  then  tries  out  the  change  (e.g.,  with  a 
sample  of  units)  and  evaluates  the  results  by  the  defined  criteria.  The  PAT  may  then  accept  the 
change  as  meeting  the  goal  or  they  may  modify  it  and  try  again. 
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Adopting  Changes 


The  PAT  needs  to  supervise  the  installation  of  permanent  changes,  and  then  monitor  their 
effects  to  confirm  that  the  results  continue.  Minor  problems  or  by-products  may  emerge,  and  the 
change  may  have  to  be  refined  and  standardized.  The  change  will  also  require  associated 
changes  in  training  for  all  affected  positions.  The  PAT  should  document  and  publicize  the  long- 
run  impacts  of  improvements  in  terms  of  savings,  return  on  investment,  or  cost/benefit  ratios. 

Central  Authority 

Certain  problems  and  solutions  transcend  the  local  installation  and  CCTT  site.  Changes 
or  innovations  in  procedures  or  TSPs  required  by  hardware  and  software  upgrades,  or  new 
equipment  and  doctrine,  apply  to  all  sites.  Some  central  agency  must  have  authority  to 
coordinate  and  disseminate  common  changes  across  sites.  Such  an  agency  would  be  responsible 
for  collecting  and  distributing  the  best  quality  improvements  developed  at  local  sites.  To  flesh 
out  the  standard  TSP  library,  all  sites  could  provide  modified  or  new  TSPs  developed  by  local 
units  that  prove  to  be  generally  useful.  This  agency  also  would  identify  unmet  training  needs, 
and  define  requirements  for  hardware  and  software  upgrades  and  additions. 

Quality  Indicators 

Our  experience  with  the  VTP  (Shlechter,  Bessemer,  Nesselroade,  &  Anthony,  1995; 
Shlechter,  Kraemer,  Bessemer,  &  Burnside,  1996)  and  the  formative  evaluation  of  the  CCTT 
TSPs  has  developed  an  appreciation  for  some  events  and  conditions  that  contribute  to  successful 
training.  Process  indicators  like  those  shown  in  Table  2  may  be  valuable  for  monitoring  some 
aspects  of  CCTT  training.  However,  more  detailed  conceptual  analysis  is  needed  to  define 
measures  of  operational  performance  for  the  total  training  system  that  can  be  accepted  as  direct 
indices  of  training  quality.  Designs  for  future  TSPs  should  include  data  collection  procedures  for 
indicators  that  support  quality  management  methods.  For  example,  the  0/C  could  provide  salient 
facts  about  exercise  execution  by  filling  out  a  brief  form  provided  with  each  Exercise  Guide. 

Supporting  Tools 

Future  simulation  developments  should  consider  requirements  for  data  collection  and 
analysis  that  support  quality  management.  Current  acquisition  policy  for  training  devices  does 
not  contemplate  such  requirements,  nor  does  it  consider  data  collection  to  be  part  of  contractor 
logistic  support.  Responsible  agencies  should  make  necessary  changes  in  both  philosophy  and 
regulation  to  enable  better  long-term  management  of  simulation-based  training. 

Commercial  off-the-shelf  (COTS)  software  can  meet  most  needs,  but  this  software  should 
be  integrated  and  interactive  with  the  simulator  system.  The  CCTT  site  needs  other  tools  to 
support  information  distribution,  scheduling,  coordination,  and  collection  of  administrative  data. 
For  example,  an  Internet  Web  site  could  be  set  up  to  provide  efficient  user  access  and  interaction 
with  the  CCTT  site  for  remote  ARNG  or  Reserve  units. 
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Table  2 


Quality  Indicators  in  Training  Phases 


Pre-Exercise  Phase 

•  Unit  training  needs  and  exercises  selected. 

•  Lead  times  completing  unit  schedule. 

•  Lead  times  to  identify  0/C  and  USW  operators. 

•  Lead  times  for  distribution  of  training  guides. 

•  Unit  leaders’  time  spent  studying  guidance. 

•  0/C  and  USW  operators’  time  studying  guidance. 

•  Times  used  in  troop  leading  and  rehearsals. 

Preparation  Phase 

•  Time  used  in  ICW  training. 

•  Time  used  in  unit’s  Familiarization  Exercise. 

•  Attendance  in  the  Familiarization  Exercise. 

•  Time  used  in  the  Practical  Exercise 

•  Personnel  participating  in  the  Practical  Exercise 

Exercise  Training  Phase 

•  Number  of  exercises  and  proportion  repeated. 

•  Total  unit  on-site  time  and  training  time. 

•  Time  to  complete  preview,  execution,  and  AAR. 

•  Quality  and  completeness  of  leader’s  brief  back. 

•  Time  and  distance  traveled  in  leader’s  reconnaissance. 


•  Proportion  of  planned  exercise  tasks  occurring. 

•  Coaching  or  other  Event  Guide  variations  required. 

•  Proportion  of  trained  tasks  discussed  in  AAR. 

•  Proportion  of  unit  participating  in  the  AAR  discussion. 

•  Delays  waiting  for  visual  or  radio  replays. 

•  Attendance  in  preview,  execution,  and  AAR. 

•  Personnel  turnover  across  exercises. 

Post-Exercise  Phase 

•  Time  spent  in  report  preparation  by  the  0/C. 

•  Interval  between  training  and  report  delivery. 

•  Who  saw  and  used  the  report  in  the  unit. 

•  Unit  training  done  in  the  next  quarter. 

Outcome  Measures 

•  Gain  in  performance  between  first  and  later  performance 
of  repeated  tasks  and  sub-tasks. 

•  Gain  in  unit  and  leader  proficiency  rated  before  and  after 
training  by  participants  and  the  0/C. 

•  Effect  on  performance  in  initial  field  exercises  compared 
to  untrained  baseline  or  control  units. 

•  User  satisfaction  with  training  site  services. 


For  quality  management,  the  simulator  system  should  provide  flexible  tools  for  on-line 
observer  or  operator  data  entry.  Computers  with  database,  statistics,  and  process  control 
software  should  network  with  the  simulator  system  to  download  indicator  data,  and  network  with 
office  computers  to  feed  preparation  of  presentations  and  reports.  Proper  equipment  can  simplify 
survey  data  entry.  Multi-media  editing  capabilities  may  enable  advanced  graphic  analysis 
methods  more  tactically  revealing  than  the  statistical  methods  now  used,  as  well  as  contributing 
to  more  vivid,  effective  AARs  and  Post-Exercise  reports. 

Wilkinson  (1997)  identified  several  types  of  tools  needed  to  aid  commanders  and  staff  to 
make  the  best  possible  use  of  the  CCTT.  One  should  provide  an  informative  overview  of  CCTT 
and  a  means  of  exploring  in  greater  depth  the  capabilities  and  limitations  of  the  CCTT  in  relation 
to  doctrine  and  tactics,  techniques,  and  procedures.  This  tool  would  provide  the  unit  guidance  on 
what  training  to  conduct  in  CCTT  in  preference  to  other  alternatives.  A  second  tool  would 
provide  software  support  for  retrieving  existing  exercises  and  TSP  materials  and  modifying 
them,  or  developing  new  exercises  and  materials  as  needed  to  meet  the  unit’s  training  needs.  A 
third  tool  would  assist  the  user  to  set  up  and  check  out  the  workability  of  the  data  input  files  and 
system  initialization  files  required  for  executing  a  planned  CCTT  exercise.  All  of  these  functions 
are  to  be  provided  by  software  in  the  “Commander’s  Integrated  Training  Tool”  being  developed 
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to  support  user’s  preparation  for  unit  training  with  CCTT  (U.S.  Army  Research  Institute  for  the 
Behavioral  and  Social  Sciences,  1997). 


Conclusion 

This  report  advanced  the  hypothesis  that  the  U.S.  Army  TAQ  program  can  serve  as  a 
workable  model  of  management  o\'ersight  for  sustaining  and  improving  simulator-based  training. 
Managers  and  trainers  should  test  this  hypothesis  with  the  CCTT  and  other  existing  simulator 
systems  and  training  programs.  If  the  results  prove  beneficial,  acquisition  requirements  for  new 
training  simulators  should  provide  support  for  quality  management  mechanisms.  Long-term 
management  functions  and  tools  have  been  neglected  in  the  current  DIS  standard  for  Exercise 
Management  and  Feedback,  and  deserve  consideration  when  the  standard  is  revised  or  expanded. 
As  new  standards  for  HLA  are  developed,  the  FEDEP  should  include  the  sustainment  and 
improvement  of  training  applications. 
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